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Abstract 

This paper describes an efficient implementation of binning for the relay channel using low-density parity- 
check (LDPC) codes. We devise bilayer LDPC codes to approach the theoretically promised rate of the decode-and- 
forward relaying strategy by incorporating relay-generated information bits in specially designed bilayer graphical 
code structures. While conventional LDPC codes are sensitively tuned to operate efficiently at a certain channel 
parameter, the proposed bilayer LDPC codes are capable of working at two different channel parameters and two 
different rates: that at the relay and at the destination. To analyze the performance of bilayer LDPC codes, bilayer 
density evolution is devised as an extension of the standard density evolution algorithm. Based on bilayer density 
evolution, a design methodology is developed for the bilayer codes in which the degree distribution is iteratively 
improved using linear programming. Further, in order to approach the theoretical decode-and-forward rate for a 
wide range of channel parameters, this paper proposes two different forms bilayer codes, the bilayer-expurgated and 
bilayer-lengthened codes. It is demonstrated that a properly designed bilayer LDPC code can achieve an asymptotic 
infinite-length threshold within 0.24 dB gap to the Shannon limits of two different channels simultaneously for 
a wide range of channel parameters. By practical code construction, finite-length bilayer codes are shown to be 
able to approach within a 0.6 dB gap to the theoretical decode-and-forward rate of the relay channel at a block 
length of 10^ and a bit-error probability (BER) of 10^^. Finally, it is demonstrated that a generalized version of 
the proposed bilayer code construction is applicable to relay networks with multiple relays. 
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I. Introduction 

Low-density parity-check (LDPC) codes have proved to be very powerful in approaching the capacity 
of conventional single-user communication channels. The key idea of LDPC codes is to practically 
implement the random coding theorem of Shannon by enforcing a set of random parity-check constraints 
on information bits. While random coding is a fundamental element of single-user information theory, 
binning is of fundamental importance in multiuser scenarios. In this paper, we explore the possibility of 
using LDPC codes to practically implement binning and to approach the theoretical results derived by 
random binning and random coding arguments for an important example of multi-user channels: the relay 
channel. 

In a relay channel, a single source X attempts to communicate to a single destination Y with the 
help of a relay. The relay receives Yi and sends out Xi based on Yi. The relay channel is defined by 
the joint distribution p{y,yi\x,Xi). A schematic of the relay channel is illustrated in Fig. 1. Although 
the capacity of the relay channel is still an open problem, several clever methods have been designed to 
take advantage of the information available at the relay. The classic work of Cover and El Gamal [1] 
describes two basic strategies: first, a decode-and-forward strategy in which the relay completely decodes 
the transmitted message and partially forwards the decoded message using a binning technique to allow 
the complete resolution of the message at the decoder, and second, a more complex quantize-and-forward 
strategy in which the relay does not need to decode the source's message. Both strategies rely on a block- 
Markov coding scheme in which each coding block consists of simultaneous decoding (or quantizing) of 
the current block at the relay and the decoding of the previous block at the destination. Cover and El 
Gamal [1] proved that the decode-and-forward strategy is capacity achieving for a class of degraded relay 
channels. 

This paper focuses on practical implementation of the decode-and-forward strategy for the relay channel. 
We restrict our attention to Gaussian relay channels at low signal-to-noise ratios (SNRs) for which binary 
linear codes are suitable. We show that, within a linear coding framework, the binning strategy in which 
a bin index of the codeword is transmitted by the relay to the destination can be interpreted as a parity- 
forwarding scheme. Further, the optimal code design for the decode-and-forward strategy entails to the 
design of a LDPC code working at two different channel SNRs: a high SNR at the relay and a low SNR 
at the destination. This represents novel LDPC code constructions, named bilayer LDPC codes in this 
paper. 

The main results of our work are as follows. We propose two new ensembles of LDPC codes, bilayer- 
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expurgated codes and bilayer-lengthened LDPC codes, both with an embedding structure, to simultaneously 
approach the capacities of two Gaussian channels at two different SNRs. The performance analysis and 
design methodologies for these new ensembles of bilayer LDPC codes are developed by generalizing 
density evolution [2] for standard LDPC codes to bilayer codes. We develop a design technique based on 
linear programming to optimize the variable degrees of the bilayer code. The two forms of bilayer code 
structure are necessary in order to accommodate the optimization of check degrees. Together, we show 
that specially structured irregular bilayer LDPC code with carefully chosen variable and check degree 
sequences can approach the theoretical decode-and-forward rate of the relay channel to within a fraction 
of dB for a wide range of different channel conditions. Finally, we generalize our code design for relay 
networks with multiple relays and show that our approach is applicable to general networks. 

The names bilayer-expurgated and bilayer-lengthened codes come from the linear coding terminol- 
ogy where expurgating refers to reducing the codebook size by increasing the number of parity-check 
equations while keeping the codeword length fixed, and lengthening refers to increasing the codebook 
size by increasing the codeword length while keeping the number of parity-check equations same [3]. 
In expurgating, codewords that do not satisfy the extra check equations are deleted from the codebook. 
The decode-and-forward operation in a relay channel can be thought of as an expurgating process in 
which the relay decodes the source codeword, then expurgates the source codebook by transmitting the 
extra parity bits (corresponding to the decoded codeword) to the destination. The code design problem 
for decode-and-forward then amounts to designing a code that is capacity-approaching at two different 
rates, both before and after expurgation. 

Alternatively, the decode-and-forward operation can also be implemented using the code lengthening 
process, in which the lengthened code is decoded at the relay, and the extra variable bits are transmitted 
to the destination. The destination then decodes a shortened code at a lower rate. Again, the code design 
problem amounts to designing a code which is capacity-approaching at two different rates, before and 
after the lengthening process. 

This paper focuses on novel design methodologies for bilayer-expurgated and bilayer-lengthened codes. 
As illustrated later, depending on the relative SNRs at the relay and at the destination, the expurgated 
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code may be more appropriate than the lengthened code, or vice versa. Together, these two techniques 
are capable of covering a wide range of channel SNRs. 

A. Related Work 

Recent interests in wireless ad-hoc and sensor networks have fueled a new surge of research activities 
both on the capacity (e.g. [4], [5], [6], [7]) and on practical implementation of communication schemes 
(e.g. [8], [9], [10]) for the relay channel in the wireless setting. Earlier work on coding for the relay channel 
includes the implementation of the decode-and-forward strategy using LDPC codes [11] and turbo codes 
[12], where performances approaching 1-1. 5dB of the theoretical limit have been reported. The work 
of [11] employs the so-called regular encoding method for the relay channel. In this case, the source 
encodes its message using a LDPC code; the relay decodes the source message and retransmits the entire 
source codeword using a second LDPC code; the destination decodes the source message jointly over 
the combined graph. Our approach is different in that an irregular coding method based on the original 
binning technique of Cover and El Gamal is implemented. In this case, the relay forwards the bin index 
of the source codeword to the destination. Although both regular and irregular coding can achieve the 
full decode-and-forward rate, irregular coding is more flexible. For example, it can be applied when the 
relay-destination link is a digital link; it is also more easily generalizable to relay networks with multiple 
relays, as shown later in Section VII; see also [13]. The LDPC code design problem for regular coding 
has recently been considered in [14]. The present work deals with the code design problem for irregular 
coding. 

This work is also related to a large number of recent and independent work on the application of LDPC 
codes to full duplex and half duplex relay channels [15], [16], [17]. In [15], an irregular expurgated coding 
protocol is devised, and a density evolution approach is used for code design. The code analysis and design 
methodology of [15] is based on conventional density evolution, where the performance of bilayer codes 
are approximated by that of the ensemble of standard LDPC codes. Using such an approximation, [15] 
reports a gap of IdB to capacity. The work of [16] considers the use of independent source and relay 
codebooks in a relay channel, which is applicable only when the relay has excess power. In this case, the 
successful decoding of the expurgated code at the destination is easy to guarantee, and the optimal design 
of the bilayer-expurgated code simplifies to that of a conventional LDPC code. The work [17] applies 
conventional LDPC codes to the half-duplex relay channel and proposes a random puncturing scheme for 
LDPC codes optimized for single-user channels. 
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The present work differs from [15], [16], [17] in that bilayer LDPC codes capable of approaching 
capacities at two different rates are explicitly designed by expurgating and lengthening conventional 
LDPC codes in a structured manner. We show that in order to truly approach the decode-and-forward 
rate for the relay channel, new ensembles of LDPC codes with a carefully designed bilayer structure 
should be used. For bilayer-expurgated LDPC codes, the random code ensemble is parameterized by 
doubly-indexed degree distribution sequences on the variables; for bilayer-lengthened LDPC codes the 
check degree distribution has double indices. While the use of conventional LDPC codes may approach 
the theoretical limit to about l-1.5dB, our specific design is asymptotically capable of closing the gap to 
0.24dB for a wide range of channel parameters. 

This work differs from the code design methodologies of [14], [15], [17] in that we devise an iterative 
degree-distribution update method based on linear programming for the bilayer codes. The iterative method 
optimizes the decoder output error probability based on the result of exact density evolution in every step. 
This approach is based on the design technique in [18] and is inspired by LDPC code design methods 
based on extrinsic information transfer (EXIT) charts [19], [20], [21], [22], [23], but is more accurate, 
since no Gaussian approximation is used in density evolution. By practical code construction, we show 
that the methodology is capable of approaching the decode-and-forward rates for a variety of channel 
conditions. 

The code design problem for the relay channel is related to the general concept of rateless codes, 
popularized by the invention of fountain codes [24] and, more recently, raptor codes [25], which are capable 
of approaching the capacities of binary erasure channels (BEC) irrespective of the erasure probability. 
Bilayer LDPC codes for the relay channel is similar to rateless codes in that both are capable of working 
at multiple different rates. However, the code design requirement for the relay channel is also fundamentally 
different in the following aspect. In a relay channel, the extra information for the second code is transmitted 
via a separately coded channel from the relay to the destination. In contrast, the channel model for rateless 
codes typically assumes that additional bits are sent through the same channel (thus are corrupted by the 
channel noise.) In addition, fountain codes and raptor codes are designed specifically for the BEC; whether 
practical rateless codes exist for more general channel models is still an open research issue [26]. In this 
sense, the design methodology for fountain codes and raptor codes cannot be directly applied to the 
general relay channel, (except in the erasure case, which is elaborated later in the paper.) 

Finally, the code construction proposed in this paper is also related to the use of punctured rate- 
compatible LDPC codes for incremental redundancy hybrid automatic repeat request (IR-HARQ) protocols 
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for wireless transmission channels (e.g. [27], [28], [29], [30], [31].) In the HARQ setting, additional coded 
data bits are sent when the decoder fails to decode. Again, the additional bits can be potentially corrupted 
by the same channel, resulting in a different coding problem as compared to the relay setting. Thus, the 
existing HARQ coding schemes are not directly applicable for decode-and-forward. 

B. Outline of the Paper 

The rest of this paper is organized as follows. Section II provides a brief review of Cover and El Gamal's 
decode-and-forward scheme, then proposes the bilayer LDPC code structure based on expurgating. After a 
review of density evolution and LDPC code optimization methods for conventional LDPC codes in Section 
III, bilayer density evolution and the proposed linear-programming-based design methodology for bilayer- 
expurgated LDPC codes are developed in Section IV. Section V proposes the bilayer-lengthened LDPC 
codes and describes the associated analysis and design tools. Code construction and numerical results are 
given in Section VI for a range of relay channel parameters. Section VII provides a generalization of 
bilayer codes to multiple-relay networks. Finally, concluding remarks are made in Section VIII. 

II. Coding for Decode- and-Forward 
A. Decode-and-Forward Strategy 

This section briefly reviews the decode-and-forward strategy of [1, Theorem 1]. In the block-Markov 
decode-and-forward scheme, transmissions occur in successive blocks. In each block i, the source and the 
relay send two messages to the destination: the source's data message denoted by Wi E {1,2,..., 2"^} 
(which is encoded through the random variable X) and the relay's message Si E {1, 2, ■ ■ ■ , 2"^^} (which 
is encoded through the random variable Xi.) The source rate, R, is such that the relay is able to decode 
Wi with an arbitrarily low error probability; however, the destination is unable to uniquely decode Wi 
because of its poorer channel. The relay's message, Sj, helps the destination decode Wi^i in block i by 
restricting u'i_i to be inside a bin of size 2"'^^^^^). Let B = {61,62, ■ ■ ■ ,S2nRi} be a random uniform 
partition of the set {1, 2, ■ ■ ■ , 2"^} into 2"^^ bins of size 2"(^^^i). The relay's message, Si, is determined 
as the index of the bin in which Wi^i falls, i.e., Wi^i E 5^-. 

Random codebooks to transmit s and w are constructed as follows. Assume that in block i, both the 
source and the relay know s^; this is a valid assumption, since Si is determined by Wi^i. The source uses 
different codebooks for each different s,;. To encode Wi, the source utilizes a random codebook X(-\si) of 
size 2"'^ generated according to the probability distribution p (a; |a;i) and transmits the codeword X{wi\si). 
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Fig. 2. Codebook construction for the relay cliannel 

In block i, the relay sends by transmitting the codeword Xi(sj) of the random codebook Xi of size 
2"^i generated according to the probability distribution p{xi). Thus, while the relay uses an independent 
codebook to encode Si, the source codebook is doubly indexed, by both Wi and Si, in order to facilitate 
cooperative transmission of Si to the destination. A key feature of the decode- and-forward schemes is 
that the source codebook must be decodable both at the relay and at the destination. In block i, the relay 
decodes Wi and computes Sj. The destination decodes Wi_i with the help of Sj. This double decodability 
condition gives rise to a bilayer code structure. 

The decode-and- forward rate for the relay channel is computed as follows. In block i, the relay decodes 
Wi which is possible if 

R<I{X-Y,\X^). (1) 
The destination, in block ^, first decodes the relay's message Sj which is possible if 



i?i</(Xi;F). 



(2) 



Upon decoding Si, is restricted to the bin Sg^ which is of the size 2"(^~^i). Since Wi^i is encoded 
by a codebook generated according to p{x\xi), the destination can successfully decode tvi^i in block i if 
R and Ri satisfy 

R- Ri< I{X-Y\Xi). (3) 
Inequalities (1), (2), and (3) give the decode-and-forward achievable rate for the relay channel: 



R = supp(,,,^) min{/(X, X,- F), /(X; Y,\X,)} 



(4) 
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Fig. 3. The Gaussian relay channel: X is power constrained to P; Xi is power constrained to Pi. 



which is also the capacity if the channel is degraded [1, Theorem 1]. 



B. Binning for the Gaussian Relay Channel 



This paper focuses on the Gaussian relay channel 



X + Z-_ 



(5) 



Y 



X + Xi + Z2, 



(6) 



as shown in Fig. 3, where Zi ~ A/'(0, Ni) Z2 ~ A/'(0, Ni + N2) are independent Gaussian noises. 
The transmitter has a power constraint P. The relay has a power constraint Pi. For the decode-and- 
forward strategy to work, the noise variance at the relay must be smaller than the noise variance at the 
destination. This channel is not degraded, unless Z2 is a degraded version of Zi (i.e. Z2 = Zi + Z' , where 
Z' ~ A/'(0, N2) is independent of Zi.) 

For the Gaussian relay channel. Cover and El Gamal [1, Section IV] showed that the optimal codebook 
X{-\si) is additive in the sense that codewords X{wi\s.i) can be constructed via 



where X{wi) ~ A/'(0, aP) and Xi{si) ~ A/'(0, Pi) are independent codebooks of sizes 2"^ and 2"^i, 
respectively. This determines an optimal joint distribution Xi) under the power constraints. The source 
divides its total power budget P into a fraction aP for transmitting new codeword Wi and a fraction of 
(1 — a)P for cooperatively transmitting the bin index Si of the previous codeword Wj_i. The optimal a 
is determined later. 

The additive structure of the X{wi\si) makes practical construction of codes for the Gaussian relay 
channel feasible. It also makes explicit the fact that Xi and X jointly transmit Si to the destination at the 




(7) 
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same time. The decoding process goes as follows. The relay decodes Wi based on Yi. 



71 _ (y)p 

Y, = X + Z, = X{w,) + + Z,. (8) 

Since Xi{si) is known at the relay, it can be subtracted. Therefore, the successful decoding of X{wi) is 
possible if 

R < nX;Y^\X^) = ^ log (^1 + (9) 

The destination observes 



Y = X + X, + Z2 = X{w,) + I 1 + x,{si) + Z2. (10) 

The decoding of Wi takes place in two stages. First, the decoder decodes Si, the bin index of iVi^i, while 
regarding X{wi) as noise. The decoding is successful if 



With Si known, the destination now subtracts Xi{si) and proceeds with the decoding of Wi in the second 
stage. This is done in the next coding block, after the bin index s^+i is decoded. The bin index restricts 
the candidate codewords into a set of size 2"(^~^^). Thus, decoding is successful whenever 

«-«'S/(*mo = iiog(i + ^^). (12) 

Combining (9) (11) and (12), Cover and El Gamal [1, Theorem 5] derived the following achievable rate 
for the Gaussian relay channel: 

i? = maxmmj-log ^1 + _ j . - log (^1 + ) ) ■ d^) 

The maximizing a in the above expression is a = 1 if P/A'^i > P1/N2 which is the case when the optimal 
strategy is not to allocate any portion of the transmitter's power to cooperate with the relay's message. 
Thus, no coherent transmission is needed between the relay and the transmitter. When P1/N2 < P/Ni, 
the optimal a is obtained by equating the two rate expressions in (13). This is when coherent transmission 
of Si by the source and the relay is beneficial. 

The code construction problem to implement the above scheme can be formulated as two subproblems: 
two codebooks are needed, Xi of rate Ri, and X of rate R. The relay's codebook, Xi, can be constructed 
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Ri = I{Xi;Y) = R+ - 

V 

Fig. 4. The code construction problem for decode-and-forward corresponds to two subproblems: constructing a source codebook to 
simultaneously approach rates i?+ and _R_, and constructing a conventional relay codebook to approach the rate R\ = _R+ — 




as a conventional error-correcting code that guarantees successful decoding at rate at the destination. 
The source's codebook, X, needs to be constructed so that the relay may decode at 

SNR, = ^ (14) 

and the destination may decode under a different SNR 

aP 

SNR_ = — , (15) 

iVi + iV2 

but with the help of extra bin index information at rate -Ri. 

The code construction problem for the Gaussian relay channel is abstracted in a schematic depicted in 
Fig. 4. The source's codebook should be a capacity-achieving code over an abstracted Gaussian channel, 
representing the source-relay link, with SNR = SNR_|_ and rate 

i?+ = ilogfl + ^) (16) 



2 ° V ^1 

which corresponds to (9). The source's codebook, with the help of relay, should also be a capacity 
achieving code over an abstracted Gaussian channel, representing the source-destination link, with SNR = 
SNR_ and rate 

\f aP , 

R_ = -\og 1 + -——- . (17) 



2 ° V N1 + N2 

which corresponds to (12). 



Note that in the regime where the optimal a = 1, the decode-and-forward rate is limited by i.e., 
extra power Pi at the relay terminal does not improve the overall rate. However, from a code construction 
point of view, extra power at the relay node simplifies the code construction problem since the relay- 
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destination link can provide more redundancy bits than necessary to the destination' This is the case 
considered in [16]. In this paper, we focus on the more stringent situation with Ri = i?+ — R_, which 
corresponds to the case P/Ni < P1/N2 with optimum cooperation factor a < 1. 

Although the coding construction in this section is derived from a full-duplex relay channel, the coding 
scheme is applicable more generally to any relay-destination channel, including a digital relay-destination 
link and the half duplex relay channel, as long as the relay-destination rate, Ri, satisfies Ri > i?+ — 

This paper focuses on the practical design of X, while ignoring multiple-access and interference- 
subtraction issues at the destination. Practical implementation of superposition coding and interference 
subtraction has been well studied in the literature. The above discussion also ignores error propagation, 
whereby an incorrect decoding of negatively affects the decoding of X{wi). In practice, the prob- 

ability of error for the decode-and-forward protocol is bounded by the maximum of failure probabilities 
of the decoding of the source's message at the relay, the decoding of relay's message at the destination, 
and the decoding of the source's message at the destination. While error propagation does not impact the 
design of capacity-achieving codes, it is practically important, especially in term of outage probability in 
a wireless fading channel. 

C. Bilayer Codes for Parity-Forwarding 

A crucial ingredient of the decode-and-forward strategy is binning. How can binning be implemented 
in practice? If we restrict our attention to Gaussian channels at low SNR (i.e. R < \) for which binary 
signaling and linear codes are optimum, then binning may be implemented by generating extra parity 
bits on the codewords of X. The generation of parity bits (or syndromes) is a natural way of partitioning 
a linear codebook into bins, with codewords in each bin satisfying a particular set of parity equations. 
The parity bits are exactly the bin indices. The idea of implementing structured binning via syndromes 
has been used in the past for Slepian-Wolf coding [32] and for channel and source coding with side 
information [33]. 

To implement binning and block-Markov coding using this idea, the relay decodes the transmitted 
codeword X{wi) in block i, and generates extra parity bits for X{wi), codes them using an independent 
codebook Xi, and sends the coded bits to the destination in the next block. The destination decodes 

'This is shown, for example, in the simulation results of [17], where a smaller gap to Shannon limit is observed when the relay is allowed 
to transmit for a longer period of time in a half-duplex relay channel. 

^However, when extra power is available at the relay node, decode-and-forward cannot be the optimal strategy. Higher rate can be obtained 
by, for example, quantize-and-forward. 
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X{wi) by utilizing the extra parity bits. Therefore, the decode-and-forward strategy is a parity-forwarding 
strategy. 

We focus on the design of a new LDPC code structure for X to implement the parity-forwarding strategy. 
As mentioned earlier, Xi can be designed as a conventional LDPC code. However, special considerations 
are needed for the design of X. Let X he a linear {n, n — ki) LDPC code with a rate of {n — ki)/n. The 
codebook X should be a capacity approaching code for the channel between the source and the relay, 
i.e., at SNR+ with a rate approaching (16). 

Let k2 be the number of randomly-generated extra parity bits for a source codeword X{wi) generated 
by the relay and provided to the destination. Then, X{wi), should satisfy two sets of parities: ki zero 
parities enforced by the source's codebook, and k2 extra presumably nonzero parity bits provided by the 
relay. Thus, a subcode of X that satisfies the additional k2 parity checks should form a {n,n — ki — k2) 
capacity-approaching code for decoding at the destination, i.e. at SNR_ with a rate approaching (17). 

The decoding of the subcode of X with k2 nonzero parity bits can be done in the exact same way as 
the decoding of a conventional LDPC code. Note that different /c2 bin index values correspond to different 
subcode; they form a coset partition of X and are related to each other through a coset leader. For linear 
codes, the subcodes can be generated by the binary addition of the coset leader to the subcode defined by 
enforcing both ki and k2 parity bits to zero. Since the subcodes are identical to each other geometrically, 
we only need to ensure that the subcode represented by the zero-codeword coset leader is well designed. 
The implementation of the parity-forwarding scheme using LDPC codes is graphically depicted in Fig. 5. 

The proposed LDPC code structure is shown in Fig. 6. We call the proposed code structure a bilayer- 
expurgated LDPC code, as the first (lower) layer corresponds to a {n,n — ki) code (for the source-relay 
channel) consisting of the ki zero parity bits and a second (upper) layer consists of the k2 extra parity bits 
which modifies the first layer in a way that a {n,n — ki — ^2) subcode represented by the overall graph 
is suitable for the source-destination channel. The overall graph represents an expurgated subcode of the 
lower-layer code. Note that the performance of a practical bilayer code is characterized by two gaps to 
capacity, that at SNR+ and at SNR_. 

D. Coding for the Erasure Relay Channel 

Before considering the design of bilayer codes for the Gaussian channel, it is useful ask whether such 
a capacity-achieving bilayer code exists in theory. It is well known that random linear codes are capacity- 
achieving for the binary symmetric channel (and for the Gaussian channel at a low SNR) under the 
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k2 




Fig. 5. Parity-forwarding implementation of decode-and-forward using LDPC codes: (a) The source message is encoded using an (n, ?i — fci) 
LDPC code; (b) The relay decodes the source's codeword; (c) The relay then generates k2 extra parity bits; (d) The k2 parity bits are 
transmitted to the destination using a separate codebook; (e) The destination first decodes the extra k2 parity bits, then decodes the source 
message over the bilayer code by searching for a codeword that satisfies k\ zero parity bits and k2 nonzero parity bits. 

maximum likelihood decoding. A subcode of a random code is also a random code. Therefore, under the 
maximum likelihood decoding, a bilayer code can be found to achieve capacities at two different SNRs. 

The question becomes more interesting if we consider practical iterative decoding methods. In this 
realm, theoretical results are available only for the binary erasure channel, for which capacity-achieving 
degree sequences for low-density parity-check codes under iterative decoding methods have been identified 
[34] [35] [36]. 

Consider the binary erasure relay channel as shown in Fig. 7, for which the source-relay channel is a BEC 
with erasure probability ei, the source-destination channel is an independent BEC with erasure probability 
£2 > ei, and a relay-destination channel is a digital link with capacity Ri. When Ri < (1 — ei) — (1 — 62), 
the capacity of this channel is known to be C = i?i + (1 — 62) [37]; decode-and-forward strategy achieves 
the capacity on this channel. 

Can practical codes achieve the capacity for binary erasure relay channel? In a remarkable development 
motivated by lossy Internet packet transmission applications, Luby [24] showed that instead of using 
conventional LDPC codes, where codewords satisfying a set of parity constraints are transmitted through 
the channel, it is possible to devise universal low-density generator-matrix (LDGM) codes, termed LT 
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Fig. 6. The bilayer-expurgated code. The lower subgraph represents a LDPC code for source-relay channel. The overall graph represents 
a LDPC code for the destination. 
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Fig. 7. The binary-erasure relay channel with a digital link from the relay to destination 



code, to achieve the BEC capacity for any arbitrary erasure probability. In the LT code construction, 
random parities generated from a carefully chosen degree distribution are transmitted through the BEC. 
Luby proved that using a parity generation function of average degree 0(ln(m)), one only needs m + o{m) 
parities to decode the transmitted bit sequence with high probability. Thus, as m oo, one can approach 
the BEC capacity regardless of the erasure probability. 

LT codes can be easily adapted to create capacity-achieving codes for the erasure relay channel. Instead 
of using LT codes as an online code, consider a block code with m source bits and n + o{n) encoded 
parity bits sent by X, with rate ^ = Ri + {1 — 62). Since Ri < (1 — ei) — (1 — 62), the relay would 
receive a sufficient number of parities to decode the source bits with a high probability. The relay then 
independently re-encodes the source bits using the same degree distribution, and sends the additional 
parities to the destination via the digital link at rate Ri. The total number of independent parities at the 
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Fig. 8. Schematic diagram of a LDPC code of rate 1 — fc/n with n variable nodes and k check nodes. 

destination is then (1 — e2)n bits from the source plus Rin bits from the relay. Thus, decoding would be 
successful for arbitrary rates below C = i?i + (1 — 62). 

The above argument shows that practical capacity-achieving codes exists for the erasure relay channel. 
In fact, the above scheme can be further improved in practice by using Raptor codes [25] instead of LT 
codes to achieve linear-time encoding and decoding performance. However, as mentioned earlier, neither 
Raptor codes nor LT codes can be used to achieve the Gaussian relay channel capacity, as capacity- 
achieving rateless codes for general binary symmetric channels have not been found [26]. The rest of the 
paper focuses on LDPC code design methods for the Gaussian relay channel that come very close to the 
best achievable decode-and-forward rate. 

in. LDPC Code Preliminaries 

This section reviews the terminologies and design methods for LDPC codes. The design methodology 
of bilayer LDPC codes for the relay channel is described in the next section and is based on the design 
procedure described here. 

A. Terminologies 

The LDPC code, originally invented by Gallager [22], is a powerful class of linear codes that can 
approach very close to the Shannon capacity of a Gaussian channel [38]. A LDPC code can be described 
by a bipartite graph consisting of two sets of nodes: variable nodes and check nodes. In a binary LDPC 
code, variable nodes take ±1 values and a check node requires the module 2 sum of all variable nodes 
connected to it to be zero. A schematic diagram of a binary LDPC code is shown in Fig. 8, where variable 
nodes are represented by circles and check nodes are represented by squares. 

The variable degree of a variable node is the number of edges connected to the node. The check degree 
of a check node is defined similarly. A code is said to have a regular variable (or check) degree if the 
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degrees of all variable (or check) nodes are equal. A regular LDPC code, originally invented by Gallager, 
is a LDPC code with both regular variable and regular check degrees. In an irregular LDPC code invented 
in [39], variable nodes or check nodes can have unequal degrees. 

An ensemble of irregular LDPC codes is described by two sets of parameters: the variable degree 
distribution > 2, and the check degree distribution pj,j > 2. The variable and check degree 
distributions define the percentage of edges in the graph that are connected to various variable and check 
degrees. Equivalently, Aj denotes the probability that an edge has a variable degree i, pj denotes the 
probability that an edge has a check degree j. Note that X]t>2 ^ ^j>2 Pj ^ ^■ 

The decoding of a LDPC code is commonly based on a message passing algorithm over the edges of 
the bipartite graph. In a message-passing algorithm, a message corresponds to an edge and represents an 
estimate for the value of the variable node connected to the edge, possibly along with a soft reliability 
information for that estimate. Messages are iteratively updated by variable updates and check updates. 
Different message-passing algorithms are possible depending on the types of messages and variable and 
check updates. 

In this paper, we focus on the sum-product algorithm, also known as the belief propagation algorithm, 
in which the message corresponding to an edge is the log-likelihood value of the variable node connected 
to it. (See [2] or [40] for further details.) Associated with such a message is a hard decision. A message 
is correct/incorrect if its associated hard decision is correct/incorrect. The message-error-probability at 
a decoding iteration is defined as the percentage of incorrect messages in the graph in that decoding 
iteration. 

The performance of an infinite-length LDPC code can be accurately predicted based on the variable and 
check degree distributions from which the code ensemble is generated. Density evolution is a tool invented 
by Gallager [22] and substantially extended by Richardson and Urbanke [2] to analyze the performance 
of a LDPC code under message-passing decoding. Density evolution tracks the evolution of the message 
probability density function (PDF) as variable and check updates are performed in successive decoding 
iterations. Details of density evolution algorithm can be found in [2]. 

It has been demonstrated that an ensemble of LDPC code exhibits a threshold phenomenon [2] . When 
decoded over a Gaussian channel with SNR > SNRthreshoid, the decoding error probability arbitrarily 
approaches zero as the length of the code increases. Conversely, if SNR < SNRthreshoid the decoding error 
probability is bounded away from zero by a positive constant regardless of the length of the code and 
number of decoding iterations. The threshold SNRthreshoid is called the convergence threshold of the code. 
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B. LDPC Code Design 

The rate of a LDPC code is determined by the variable and check degree distributions as follows. 
Let E be the total number of edges in the graph. Then, the total number of variable nodes is given by 
-E^.^gAj/i; the total number of check nodes in the graph is given by EJ2j>2Pj/j- Thus, the rate of 
the code is given by: 

i?=l-p^. (18) 

The LDPC code design problem is to find a pair of variable and check degree distributions that maximize 
(18) while ensuring successful decoding at a given SNR. 

It has been shown that for certain classes of decoding algorithms, the optimum check degree distribution 
is concentrated around a mean value and a capacity approaching LDPC code has only one or two 
consecutive check degrees^ [39, Section 3.3]. As a result, in LDPC design, it is common to fix a regular 
check degree and optimize the code over the variable degree distribution only. The optimum check degree 
is often found by trying different values. Empirically, the optimum check degree of the code is found to 
increase with the maximum allowed variable degree, max(d^), and with the rate of the code. Note that 
the optimal check degree for codes at different rates can be very different. This fact will be important 
later for the design of bilayer codes, which have to operate at two different rates. 

With a fixed check degree, the rate maximization problem (18) is equivalent to the maximization of 
J2i>2 ^ih- This leads to a linear programming approach to code optimization. Linear programming method 
for variable-degree optimization first appeared in [39]; the design method was later modified in [38]. The 
method used in this paper is based on a different approach devised in [18], which is inspired by EXIT-chart 
based methods. The rest of this section outlines this approach. 

Assuming a fixed check-degree distribution, the basic idea is to start with some variable-degree dis- 
tribution Aj, then iteratively improve the overall rate while ensuring convergence by identifying a better 
A.. 

For a fixed A^, the iterative decoding process can be characterized by a set of message PDF at the 
beginning of each decoding iteration, denoted as p', / = 1 ■ ■ - L, where L is the maximum number of 
iterations. Here, each decoding iteration is defined to be a check update followed by a variable update 
for all messages in the graph (i.e. a parallel message-passing schedule is assumed.) The message PDF at 
beginning the (/ + l)'th decoding iteration, can be computed using the density evolution algorithm 
in [2] or the discretized density evolution scheme of [38]. The message error probability of each iteration 

^See, for example, a database of optimized LDPC codes available at [41]. 
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can be directly calculated from pK 

The main idea is to start with some initial Aj (which determines a set of p''s), then assume that j»''s are 
fixed and incrementally adjust Aj to maximize the overall rate while ensuring convergence. This incremental 
adjustment is, of course, not exact, as depends on Aj. However, as we shall see, such an assumption 
yields a low-complexity, yet accurate, code optimization procedure if the incremental adjustment on Aj is 
sufficiently small, which implies that the change in is also small. 

Fixing p'', the incremental adjustment on A/ can be done via linear programming using an approach 
inspired by EXIT charts [19], [20], [21], [22]. The key ingredient is to define a set of error profile 
functions an irregular LDPC code, ej(p), as a function of the input message PDF, for each variable degree 
i separately, as follows. Consider an auxiliary LDPC code with regular variable degree i,i > 2, and the 
same check degree as the irregular code. The degree i error profile, ej(p), as a function of input message 
density p, is defined as the message error probability after one density-evolution iteration in the auxiliary 
regular LDPC code, with an initial message PDF p. This is closely related to the concept of elementary 
EXIT chart defined in [23]. The difference is that exact density evolution is used; there is no Gaussian 
approximation of message densities. 

Let e(p'+^) denote the message error probability corresponding to the message density p'^^ in the 
(/ + l)'th decoding iteration in the original irregular LDPC code. The degree i error profile, ei(-), can be 
used to compute e(p'"*"^) for the irregular code as follows: 

e(p'+^) = 5^A.e.(p^ 

j>2 

The message-passing decoding algorithm converges if the message error probability of the code decreases 
with each decoding iteration. This can be formulated by a set of convergence inequalities as follows: 

e{p'^') = J2>^Mp')<<p'), l = l---L. (19) 

i>2 

The above set of inequalities is linear in Ai,z > 2, if j»''s are fixed. However, j)''s depend nonlinearly 
on Aj. Nevertheless, (19) can still be used to formulate an approximate linear programming problem to 
update Aj. The idea is to update Aj slowly by enforcing a more stringent convergence condition 

5^Aje,(p') </ie(pO, l = l---L (20) 

where is a convergence factor that increases slowly from to 1 in the iterative design process. This 
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works because a small change in /i corresponds to only a small change in the convergence behavior of 
the code, and thus a small change in error profile ej(p'). 

Using (20), an iterative optimization scheme for updating the variable degree distribution can be 
formulated as follows. A sequence of linear programming problems 



max 

Aj,i>2 

i>2 



Y^Kh (21a) 

i>2 

s.t. 5^V,(p') </e(p'), / = 1---L (21b) 

i>2 

^A, = l (21c) 

i>2 

are solved successively, where h denotes the optimization iteration number and / is decoding iteration 
number. We start with all variable degrees set to max((it,), i.e. Aniax{d^) = 1, and use this Aj to compute 
the ej(p') coefficients in (21b). This ensures a small initial /i° (as long as an appropriate check degree is 
selected). We then solve the resulting linear programming program to obtain an updated Aj. For this Aj, 
we recompute ei(p'), and solve the linear programming problem again with a slightly increased fJ'. The 
slight increase in jJ" ensures that the change in ej(p') is small as compared to the previous iteration. The 
new variable degree distribution, Aj, obtained from (21) is then used to update ei(p') coefficients. The 
optimization is repeated with fi^'^^, until jj!^ eventually reaches 1. 

This procedure is reminiscent of the EXIT-chart approach, because the value of /i defines the shape of 
the convergence behavior. To speed up the /i-update process, we also use a backtracking algorithm: at the 
end of the /I'th iteration, a greedy increase in ji^ is performed. If the resulting degree distribution does 
not correspond to a converging LDPC code (i.e., (21b) cannot be satisfied with jji^^^), jJ'^^ is reduced 
and the optimization is repeated. 



IV. Design of Bilayer-Expurgated LDPC Codes 

We now extend the use of iterative linear programming to design bilayer-expurgated LDPC codes for 
the relay channel. Toward this end, we first characterize the ensemble of bilayer-expurgated LDPC codes 
and devise bilayer density evolution as a performance analysis tool appropriate for this new ensemble. 
Based on bilayer density evolution, the design methodology described in the previous section is then 
adapted for the optimization of bilayer LDPC codes. 

A key simplifying assumption in our linear programming methodology for LDPC code design is that 
the check degree is concentrated, which is near optimal for the conventional LDPC codes. However, 
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concentrated check degree is difficult to realize for a bilayer-expurgated code, as two sets of parity checks 
are involved, and the code must work at two different SNRs. 

Check-degree optimization is a key aspect of bilayer code design. This paper proposes two bilayer 
LDPC code designs. The first approach, which is described in this section, assumes two concentrated 
check degrees at the two sets of parity bits of the bilayer code. The second approach defines a different 
code ensemble by lengthening a LDPC code and can be thought as a dual of the first approach. The 
design of bilayer-lengthened codes is described in the next section. 

A. Bilayer-Expurgated LDPC Code Ensemble 

The design of bilayer-expurgated LDPC codes is based on a code ensemble defined as follows. The 
bilayer graph, as shown in Fig. 6, consists of three sets of nodes and two sets of edges. The three sets 
of nodes correspond to one set of variable nodes, and two sets of check nodes: the lower check nodes 
corresponding to check nodes in the lower subgraph of Fig. 6, and the upper check nodes corresponding 
to check nodes in the upper subgraph in Fig. 6. Edges are grouped in two sets: those connecting the 
variable nodes to the lower check nodes, and those connecting variable nodes to the upper check nodes. 
We call an edge a lower edge, if it connects a variable node to a lower check node. Similarly, an upper 
edge denotes an edge belonging to the upper subgraph in Fig. 6. 

The lower degree of a variable node is defined as the number of lower edges connected to it. Likewise, 
the upper degree of a variable node is the number of upper edges connected to it. The lower degree of 
an edge is defined as the lower degree of the variable node it is connected to, and similarly the upper 
degree of an edge is the upper degree of the variable node connected to that edge. The minimum lower 
variable degree is 2 as the lower subgraph should be a valid LDPC code for the source-relay channel. 
The minimum upper variable degree is 0, since some variable nodes may not participate in any of the k2 
extra parity checks generated by the relay. A variable node is said to have degree (z, j) if it has a lower 
degree i and an upper degree j. Similarly, an edge is of degree (i, j) if it is connected to a degree («, j) 
variable node. 

We assume regular check degrees for check nodes in the lower and upper graphs. The lower check 
degree of a bilayer graph, dc, denotes the number of edges connected to check nodes in the lower subgraph. 
Likewise, the upper check degree, rf^, equals to the number of edges connected to an upper check node. 

The ensemble of bilayer LDPC codes can be characterized by a variable degree distribution, Ajj , i > 
2, j > 0, which defines the percentage of edges with lower degree i and upper degree j and a parameter 
•T] which defines the percentage of lower edges in the bilayer graph. In other words, the probability that an 
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edge is connected to a variable with lower degree i and upper degree j is given by Ajj, and the probability 
that an edge is a lower edge is given by rj. Note that J2i>2j>o ^ < rj < 1. 

Note also that a bilayer LDPC code reduces to a conventional LDPC code if Aj j = XiXj.i > 2,j > 
for some set of parameters Aj with Aq = Ai = 0. 

B. Bilayer Density Evolution 

Because the ensemble of bilayer-expurgated LDPC codes is statistically different from a conventional 
LDPC code ensemble, conventional density evolution algorithm must be modified in order to accurately 
predict the performance of the bilayer code. In the conventional density evolution analysis, the input 
message densities to all check nodes at each density evolution iteration are the same, since the probability 
that an edge emanating from a check node is connected to a degree i variable node is equal to Aj for all 
check nodes. However, in a bilayer-expurgated code there is a distinction between lower edges and upper 
edges. Therefore, in order to predict the performance of a bilayer code, evolution of two densities should 
be tracked: the lower density corresponding to the density of messages in the lower subgraph, and the 
upper density corresponding to the density of messages in the upper subgraph. 

Let and denote the message densities at the input of lower and upper check nodes in the lower 
and upper subgraphs, respectively, at the beginning of the /'th decoding iteration. The message densities 
after a check update can be computed for and using the conventional density evolution check update 
as described in [2]. Let p'' and g'' denote the evolved versions of and g' after the check updates. For 
log-likelihood message-passing decoding, the density-evolution update at a degree variable node can 
be computed from p'' and p'' to obtain the message densities, p-^^ and g-^^, as follows: 

p[y = ® i&q'^) ®Pc, i>2,j>0 (22) 

gg^ = (®y) ® (^■'■"^g") ®Pc, « > 2,j > 1 (23) 

where Pc denotes the density of the log-likelihood ratio received over the channel, and ®' denotes 
convolution of order i. (By convention, for any density /, = / and = 5, where 5 denotes 
the Dirac delta function.) The input message densities to lower and upper check nodes, at the beginning 
of the (/ + l)'th iteration can be computed as follows: 

i>2j>0 
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j>2j>0 

Note that the probability that a degree (z,j) edge is a lower edge is given by z/(z + j). 

Similar to the error profile function for a conventional LDPC code, the lower-graph degree (i, j) error 
profile function, e\j{p\ g'), is defined for a bilayer-expurgated LDPC code as the message error probability 
corresponding to the density p^^^, after one density evolution iteration with input message densities and 
qK Similarly, e^j{p\ q^) is defined as the message error probability corresponding to g'^^ after one density 
evolution iteration for input message densities p^ and qK Let e{p''^^, denote the overall message error 
probability in the bilayer graph corresponding to the message densities p'^^ and The overall message 
error probability at the beginning of the (/ + l)'th decoding iteration, e(p'"'"^, g'"*"^), can be computed as 
a linear combination of e}j{p'-,q'-) and e'^j{p'-,q^) as follows: 

i>2,i>0 ^^^^ ^ 

The above formulation allows an approximate linear programming optimization of Ajj. 

C. Bilayer-Expurgated LDPC Code Optimization 

The design of a bilayer-expurgated LDPC code involves finding a variable degree distribution Ajj , ? > 
2, j > 0, a parameter 77, and a pair of check degrees, and d'^, such that the lower subgraph represents 
a capacity-approaching LDPC code over the source-relay channel at SNR+, and the overall bilayer code 
is capacity approaching at SNR_ < SNR+. 

One way to formulate the design problem is to fix dc, d'^, and jointly optimize Aj j and 77. This approach 
is taken in our previous work [42]; it is equivalent to a joint optimization of both the lower subgraph 
and the overall graph to achieve the highest overall rate. In this paper, we utilize a different approach 
by fixing the lower-graph code to be a capacity-approaching LDPC code at SNR+ and searching for a 
variable degree distribution, Aj^, that is consistent with the lower-graph code and is capacity approaching 
at SNR_. 

We formulate the rate maximization problem for the overall code as follows. Fixing the check degrees 
dc, d[, the rate of the bilayer graph is related to the parameter 77, since 77 depends on the number of check 
nodes in the graph 
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By fixing the lower graph, i.e., fixing n, ki and the lower variable degree distribution Aj, the rate of the 
bilayer code, defined by 1 — (ki + / n, can be maximized by minimizing k2 or equivalently maximizing 
T]. The distribution Aj is related to Ajj as follows: 

A. = iV-^A,,,. (28) 

For a fixed Aj, (28) can be rewritten in a linear format in terms of A^ ^ and 77: 

V -^A,,, - r]X, = 0. (29) 

Fixing dc, d'^, and Aj, an approximate linear programming update for Aj and r] can be formulated using 
(26) to iteratively maximize r] as follows: 

max ?7 (30a) 

s.t. - T/Ai = i>2 (30b) 

i>o * 

5^ A,, f-^el/p^, q') + -^e^_^.(p', g')) < /e^', g'), / = 1 ■ ■ ■ L (30c) 

5^ A„ = l (30d) 

i>2,j>0 

where /i is the optimization iteration number, and / is the decoding iteration number. The coefficient < 
yu'' < 1 plays the same role as the jJ^ in (21) and is slightly increased at each optimization iteration toward 
1. The error profiles e\j{p\q'-), e1j{p\q^) and e{p\q^) are recomputed at the end of each optimization 
iteration using bilayer density evolution, given the new Ajj and 77. 



To initialize the above iterative optimization, an initial degree distribution, Ajj, that is consistent with the 
lower-graph degree distribution in terms of (29) and guarantees a fast decoding convergence with a small 
> should be found. Such a degree distribution can be found using a linear programming optimization 
that minimizes r], since minimizing 77 or equivalently maximizing k2 corresponds to adding as many extra 
parity bits as possible which ensures a fast decoding convergence. The initializing linear-programming 
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optimization can be cast as follows: 



mm rj 



(31a) 



s.t. 



(31b) 



*>2J>0 




I > 2 



(31c) 



To complete the design methodology of bilayer LDPC codes, we need to pick appropriate check degrees 
dc and d'^. Unfortunately, the overall bilayer code cannot always have a concentrated check degree (i.e. 
dc = d'J, as one would like. The reason is that the optimum average check degrees at SNR+ and SNR_ 
can be far apart if the gap between SNR+ and SNR_ is large. In this case, the upper check degree, d'^, 
should be small enough to compensate the effect of a large lower check degree, dc, in order to lower the 
average check degree of the bilayer code. 

An appropriate check-degree pair, dc and d'^, can be found for a bilayer code by an exhaustive search 
over a reasonable range of values for dc and d'^. In our scheme, we pick a check degree dc and find an 
optimized conventional LDPC code corresponding to the lower-graph code with regular check degree dc 
at SNR+, using the design scheme described in Section III-B. Then, we try the optimization procedure 
of (30) for various values of d'^ to find a suitable d'c- In some cases, this procedure needs to be repeated 
several times to find a satisfactory dc and d'c pair. 

As mentioned earlier, the optimum check degree for a conventional LDPC code is often concentrated 
around a fixed value. Thus, when the gap between SNR+ and SNR_ is small, the difference between the 
optimal dc and d'^ is likely to be small, and this scheme works well. However, if the gap between SNR+ 
and SNR_ is large, the optimal check degree d'c is often much smaller than dc, resulting a larger gap to 
capacity. However, in the extreme case, the optimal d'c may become 1 , in which case a new code structure 
emerges, and a new code design methodology is called for. 

Fig. 9 shows the effect of rf^ = 1 on the structure of the bilayer-expurgated graph. The lower graph in 
this case is split into two parts: variable nodes connected to the upper checks on the left, and all other 
variable nodes on the right. A regular check degree rf^ = 1, in effect, removes the variable nodes in the 
upper part of the graph, by completely determining their values. This completely changes the structure of 
the bilayer graph. In the next section, we consider the code ensemble corresponding to this new graph, 
and develop new design methodology and the analysis tools for it. 
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Fig. 9. The effect of = 1 on the bilayer-expurgated graph of Fig. 6. The lower graph is split into two sections. 



V. Bilayer-Lengthened LDPC Code 



By removing the upper edges in Fig. 9, the variable nodes of a bilayer-expurgated LDPC code are split 
into two parts. In this case, it is natural to consider a new bilayer LDPC code ensemble, in which different 
variable-degree distributions are assigned to the two groups of variable nodes. This corresponds to a code 
structure for which the overall lower graph of Fig. 9 must be capacity-achieving at SNR_, while the right 
part of the lower graph must be capacity-achieving at SNR+. This new code is schematically shown in 
Fig. 10. The overall lower code can be thought of as a lengthened version of the right part of the lower 
code. This new ensemble is named bilayer-lengthened LDPC code in this paper. 

The parity-forwarding scheme with this new bilayer codes can be described as follows. The source 
encodes its data using the bilayer code corresponding to the overall graph shown in Fig. 10. Thus, each 
codeword satisfies all parity-check nodes present in the bilayer graph (in contrast to the earlier bilayer 
code in which the source encodes its data over the lower subgraph). The relay decodes the source's data 
over the bilayer graph and forwards the values of the upper variable nodes to the destination, using a 
separate codebook. (Note that the relay re-encodes variable bits, whereas, in the previous structure the 
relay re-encodes parity bits.) The destination first removes the upper part of the graph, then updates the 
value of parity-check nodes in the graph. For example, the new value of a parity-check node corresponding 
to the constraint vi + V2 + vj, + = after removing vi and f 2 would be vi + V2 corresponding to the 
constraint + = V1+V2. (The new values of check nodes play the role of a bin index for the received 
codeword at the destination.) Finally, the destination decodes the remainder of the codeword over the 
lower subgraph (in contrast to the earlier bilayer code in which the destination decodes the source's 
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codeword over the overall graph). 

The advantage of this scheme is that the check degrees are reduced after the removal of the upper 
graph. Therefore, this code is suitable for a relay channel with a large gap between the SNR+ and SNR_. 

Many of features of the bilayer-lengthened LDPC code are the dual of the bilayer-expurgated code: 
the roles of variable nodes and check nodes are interchanged in the parity-forwarding scheme; the source 
encodes its data over the lower graph in one, and over the overall graph in the other. The bilayer-expurgated 
code performs well for small gap of SNR+ and SNR_; the bilayer-lengthened code works well for larger 
gaps. 

The bilayer-expurgated code is closely related to rate-compatible LDPC codes for HARQ, which are 
often devised by randomly puncturing a high-rate code to produce low-rate codes (e.g. [29], [27], [31]). The 
design methodology proposed in this paper differs from random puncturing, as the degree distributions 
of the punctured bits and the remaining bits are explicitly designed, as shown in the next subsection. 
The bilayer-lengthened code structure considered in this paper is inspired by a code construction, called 
Matrioshka codes, introduced in [43] for the universal Slepian-Wolf source coding problem. 

A. Bilayer-Lengthened LDPC Code Ensemble 

Similar to the bilayer-expurgated code, the bilayer-lengthened graph consists of three sets of nodes and 
two sets of edges. The nodes are grouped into one set of check nodes (in contrast to the earlier bilayer 
graph in which there is one set of variable nodes), and two sets of variable nodes (in contrast to the earlier 
bilayer graph in which there are two sets of check nodes): the lower variable nodes corresponding to 
the variable nodes in the lower subgraph of Fig. 10, and the upper variable nodes corresponding to the 
variable nodes in the upper subgraph in Fig. 10. The edges are grouped in two sets: those connecting the 
check nodes to the lower variable nodes, and those connecting check nodes to the upper variable nodes. 
We call an edge a lower edge, if it connects a check node to a lower variable node. Similarly, an upper 
edge denotes an edge belonging to the upper subgraph in Fig. 10. 

Each check node in the bilayer-lengthened graph has dc edges in the lower subgraph and d'^ edges in 
the upper subgraph. Similar to a conventional LDPC code, the degree of a variable node is defined as 
the number of edges connected to it. An edge is said to have a variable degree i if it is connected to a 
variable node of degree i. 

The ensemble of bilayer-lengthened LDPC codes is defined by the lower variable degree distribution, 
the upper variable degree distribution, and two regular check degrees and d'^. The lower variable degree 
distribution, > 2, defines the percentage of lower edges of various degrees in the lower subgraph. 
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Fig. 10. The bilayer-lengthened LDPC code. The relay decodes the overall code and provides the value of upper variable nodes to the 
destination, using a separate codebook. The destination decodes the lower subgraph. 

i.e., the probability that a lower edge is connected to a degree i variable node is given by \}. Similarly, 
the upper variable-degree distribution, Af,2 > 2, describes the probability that an upper edge is of degree 
i. The lower and upper distributions, X} and Af , satisfy J2i>2 ^1 ~ X]i>2 ^"i ~ ^■ 

Note that the ensemble of bilayer-lengthened LDPC codes is not equivalent to either the conventional 
LDPC codes or the bilayer-expurgated LDPC codes discussed earlier, because in both of these earlier code 
ensembles, the variable degree distributions for all variable nodes are the same. Hence, density evolution 
tools for conventional LDPC codes and for bilayer-expurgated codes are not valid for the bilayer-lengthened 
LDPC code and should be modified. 

B. Bilayer Density Evolution 

The densities of messages over lower and upper edges are in general not equal at each decoding 
iteration in a bilayer-lengthened LDPC code. This is because the lower and upper edges have different 
variable degree distributions. Thus, similar to the case of the bilayer-expurgated LDPC codes, to predict 
the performance of an infinite-length bilayer-lengthened LDPC code, we need to track the evolutions of 
two densities in the upper and lower subgraphs of the lengthened graph. 

Let p'- and denote the message densities in the lower and upper parts of the graph at the beginning of 
the /'th decoding iteration. Let p'' and g'' denote the evolved version of and g' after check updates. Let 
© denote the check density-update operation as described in [2], e.g., / = /i © /2 is the output message 
density after an update at a check node of degree 3. Then, the output message density at a check node 
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of degree d with input message density / can be computed as (B'^^^f = /©/©■■■©/. Hence, p'^ and 
g'' can be computed using the check density-update operation as follows: 

/ = (©'^-"V) © (©<g'), 4 > 1 (32a) 

/ = (©V) © (©'^^-'g'), < > 1 (32b) 

and for d'^ = 1: 

/ = (©'^-V') © (g'), 4 > 1 (32c) 

q'l ^ ^^d^pi^ (32d) 

where ©V — / for any density /. 

The computation of variable density updates is straightforward using the convolution operation. Let 
p\~^^ denote the output message density after a variable update at a variable node of degree i in the lower 
subgraph, with an input message density p''. Let g-^^ denote the output message density after a variable 
update at a variable node of degree i in the upper subgraph, with an input message density q'K Using the 
convolution operation ®, we have 

= ©*-y ©p^, z>2 (33) 
g-+' = ©'-'g"®Pc, i>2 (34) 

where pc is the channel message density. 

The message densities in the lower and upper subgraphs after the variable update (i.e. at the beginning 
of (/ + l)'th decoding iteration), p'^^ and g'"*^^, can be computed from p^"*"^ and ql~^^ as follows: 

i>2 
i>2 

Let e(p'+^, g^"*"^) denote the message error probability of the message densities and g^"*"^ at the begin- 
ning of the (/+l)'th decoding iteration. Let ej(p', g') denote the message error probability corresponding to 
p'^^, which is the message density of degree-i lower nodes after one density evolution iteration with input 
message densities p^ and g'. Similarly, let e^(p',g') denote the message error probability corresponding 
to g'"*"^, which is the message density of degree-z upper nodes after one density evolution iteration with 
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input message densities and g'. The overall message error probability at the beginning of the (/ + l)'th 
iteration, e(p'+^, g'"*"^), can be found as a linear combination of eKp'-, g') and e^{p'-, g') functions as follows: 



where 77 = dc/{dc + d'J denotes the percentage of lower edges in the bilayer-lengthened graph. The 
approximate linear structure of (37) is used to form an iterative linear programming procedure to update 
the variable-degree distributions A^^ and as discussed in the next subsection. 

C. Bilayer-Lengthened LDPC Code Optimization 

The design of a bilayer-lengthened LDPC code involves finding a pair of variable degree distributions, 
Ai, {i > 2), and a pair of check degrees, dc and d'^, for the lower and upper subgraphs in the bilayer 
structure of Fig. 10, such that the overall graph is a capacity-approaching LDPC code for a Gaussian 
channel at SNR+, while the lower graph is a capacity-approaching LDPC code at SNR_. 

Similar to the previous design, we fix the check degrees dc and d'^. (Appropriate check degrees, dc 
and d'^, can be found by an exhaustive search over a reasonable range of values for dc and d'^.) We also 
fix the lower variable-degree distribution A^^ to be a capacity-approaching distribution for a conventional 
LDPC code optimized at SNR_, (which is found independently.) The design problem is now reduced 
to finding an upper variable-degree distribution Af, such that the overall lengthened graph represents a 
capacity-approaching code at SNR+. (Note that in contrast to the design problem of a bilayer-expurgated 
code, the lower-rate code is fixed here, and the higher-rate code is optimized.) 

The rate of the overall bilayer-lengthened code is 1 — k/(ni + n2), where k denotes the number of 
check nodes, ni is the number of lower variable nodes, and n2 is the number of upper variable nodes. 
The number of upper variable nodes, n2, is given by d'^kJ2i>2-^'i/^- Thus, fixing the lower-graph code 
and d[, the rate of the overall graph can be maximized by maximizing J2i>2 ^fh- To ensure convergence 
of the overall code, we make use of the error profile function (37). More specifically, fixing 77, dc, and 
d'^, the linear programming update for Af can be formulated as follows: 



e(p'+\g'+i) = $^r/A,ieKp',g') + {I - r,)Xy,{p\q'). 



(37) 



i>2 



max 

A? 




(38a) 



i>2 



S.t. 



(38b) 



i>2 




(38c) 



i>2 
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where h denotes the optimization iteration round, and / is the decoding iteration number. The new upper 
variable-degree distribution is used to update the coefficients e}{p\ g'), e^(p', g'), and e(p', g') for the next 
optimization round through bilayer density evolution. The coefficient fi'^ is slowly increased toward 1. 
This enforces an approximate local linearity condition with respect to Af, in the same way as in (20). As 
an initialization value for A^, we set Ai„ ,,-, = 1. 

The bilayer-lengthened LDPC code is a suitable code structure, if the gap between SNR+ and SNR„ 
is large. However, it has larger gaps to the capacity for smaller values of SNR_ and smaller differences 
between SNR+ and SNR_. This is because, for a fixed subgraph the minimum number of upper variable 
nodes is given hy k/ max(dy), which corresponds to an upper subgraph with regular variable degree 
max((i„) and with k check nodes each with degree d[ = 1. Thus, a small SNR_ (which implies a large k), 
leads to a large minimum additional variable nodes needed in the lengthening process, and consequently 
a larger minimum SNR_|_ — SNR_. However, by increasing max{dy), the minimum SNR_|_ — SNR_ can 
be reduced. 

When the gap between SNR+ and SNR_ is small, the bilayer-expurgated LDPC code design of Section 
IV has good performance. In fact, the rate difference can be arbitrarily small for the bilayer-expurgated 
code. Thus, the bilayer-expurgated LDPC code and the bilayer-lengthened LDPC code are complementary 
structures that cover the entire range of rates/SNRs. 

VI. Code Construction and Numerical Results 

Using the described schemes, six codes, listed in Tables I- VI, are designed for binary-input Gaussian 
channels with various noise parameters. The maximum variable degree, max((i„), for all cases, is chosen 
to be 20. To speed up the design procedure, for both bilayer-expurgated and bilayer-lengthened LDPC 
codes, discretized density evolution approach of [38] is utilized with 13-bit quantization and a maximum 
log-likelihood value 25. To verify the asymptotic infinite-length threshold, the empirical bit-error-rate 
(BER) performance curves for practically constructed codes are shown in Fig. 11. The block lengths for 
bilayer-expurgated codes are in the order of 100,000. The block lengths for bilayer-lengthened codes are 
70,000. 

Code A is constructed for a small rate difference i?+ — -R_, using the bilayer-expurgated structure. 
Code B is constructed using the bilayer-lengthened structure, which is more suitable for a large rate 
difference. Codes C and D are designed to compare the performance of expurgated and lengthened 
structures at low SNRs for target rates 0.3 and 0.4. For medium SNRs, Codes E and F are designed 
using the expurgating and lengthening structures for target rates 0.5 and 0.7. We observe that for most 
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rate pairs the lengthened structure outperforms the expurgated structure slightly. The bilayer-expurgated 
structure has a better performance for target rates that are very close to each other and at low SNRs. Over 
a wide range of SNRs, the asymptotic infinite-length threshold obtained is at most 0.24dB away from the 
theoretical limit, while finite-length BER results are within at most 0.6dB of the capacity. 

More specifically. Code A (Table I) is designed for target rates 0.65 and 0.7. Using the bilayer-expurgated 
structure, this code achieves the rate pair _R„ = 0.6363 and _R+ = 0.7000, i.e., the achieved rate is less 
than 2.1% smaller than the target rate. (The lower graph, corresponding to the higher-rate code, is 
fixed to be a conventional LDPC code with rate 0.7 from [41].) The best check-degree pair for this code 
is found by exhaustive search to be dc = 15 and d'^ = 8. The convergence threshold of the overall bilayer 
code, as predicted by bilayer density evolution, is within a 0.1727 dB gap to the theoretical limit. At 
BER = 10~^, the SNR gap to the Shannon limit of the lower rate channel is about 0.33 dB at a block 
length of 100,000. 

At a large SNR differences, the bilayer-expurgating LDPC code does not show a good performance. 
Code B (Table II) is designed for a large SNR difference of about 9 dB, using the bilayer-lengthened 
structure, for target rates 0.3 and 0.9. The achieved rate pair using the bilayer-lengthened structure is 
i?„ = 0.2871 and R+ = 0.8932, which are less than 4.2% and 0.75% below the target rates. The 
convergence thresholds, as predicted by the density evolution, are within a 0.2369 dB gap to the lower- 
rate channel capacity and within a 0.1357 dB gap to the higher-rate channel capacity. At a BER of lO"'', 
the corresponding SNR is within less than 0.25 dB gap to the Shannon limit for a block length of 70,000. 
(The lower rate component is designed using the scheme described in Section III-B.) 

To compare the performance of the bilayer-expurgated structure and the bilayer-lengthened structure. 
Codes C and Code D (Tables III and IV) are constructed using the two structures for target rates 0.3 and 
0.4. The achieved rates using the expurgating structure are 8.2% and 3.6% below the target rates i?+ and 
-R_, respectively. The achieved rates using the lengthening structure outperforms those of the expurgating 
structure and are 3.45% and 3.9% below the target rates -R+ and R-, respectively. At BER = lO"'', the 
SNR gap to the capacity of the lower rate channel for Code C is close to 0.9 dB for a block length 
of 100,000, while Code D has a less than 0.6 dB gap to the Shannon limit for a block size of 70,000. 
(The higher-rate component of the expurgating structure and the lower-rate component of the lengthening 
structure are fixed with conventional LDPC codes designed using the scheme described in Section III-B.) 

Finally, Codes E and F (Tables V and VI) are designed to compare the performance of the proposed 
expurgating and lengthening schemes at target rates 0.5 and 0.7. The achievable rates of the bilayer- 



32 



TABLE I 

Bilayer-ExpurgatedLDPC Code for SNR+=2.7330 dB SNR_ = 1.9262 dB 



Bilayer-Expurgated LDPC code (A) 












I = 2 


0.1398 


0.0408 








I = 3 


0.1323 


0.0885 








i = 6 


0.0831 











i = 7 


0.0295 


0.1332 








i = 20 








0.2600 


0.0928 


r] = 0.8982, 4 = ] 


L5, 4 = 8 


= 0.6363, Threshold gap=0.1727 dB 


R+ = 0.7000, Threshold gap=0.08474 dB 



expurgated Code E are 7.62% and 0.0% below the target rates i?+ and respectively. As a comparison, 
the bilayer-lengthened Code F achieves a rate pair within 2.46% and 1.3% below the target rates i?+ and 
-R_. Asymptotically, the convergence threshold of the bilayer component of Code E is within 0.5143 dB 
to the Shannon limit. At a block length of 100,000, the lower-rate code of Code E achieves a gap of 
0.8 dB to the Shannon limit at BER = 10"^. Code F has a convergence threshold within 0.1641 dB 
of the Shannon limit of the lower rate channel. At BER = 10~^, the SNR gap of Code F is 0.6 dB 
to the Shannon limit. For this rate pair, the lengthening structure outperforms the expurgating structure. 
(The higher-rate component of Code E is fixed as a conventional LDPC code designed using the scheme 
described in Section III-B. The lower-rate component of Code F is fixed with a conventional LDPC code 
found in [41].) 

It should be noted that only the BER performance of the bilayer structures are presented. The BER 
curves for the lower graph codes are omitted, as the lower graphs in both structures are fixed with 
conventional LDPC codes. 

VIL Multilayer LDPC Codes for Relay Networks 

Thus far, we have focused on the single-relay channel and show that bilayer LDPC codes can be 
designed to approach the best decode-and-forward rate in this classical setting. In a more general setting, 
bilayer codes (or multilayer codes) can also be adopted for multiple-relay networks. This is the subject 
of this section. 

Multiple-relay networks can have many different topologies. One way to generalize the decode-and- 
forward rate to multiple-relay networks is to impose a linear ordering on the intermediate relays, and let 
each relay completely decode the source's message with the help of relays prior to itself, then participate 



TABLE II 

Bilayer-LengthenedLDPC Code for SNR_ = -3.0728 dB and SNR+ = 5.6148 dB 



Bilayer-lengthened LDPC code (B) 




\i 

\ 


\ 


I — A 


V.jZZ / 


U. IDJ J 


i O 


9107 


9^^1 7 

\J.A\J 1 1 


I = 5 





0.1505 


1 = 6 


0.1247 





i = 7 


0.1194 





i = 10 





0.2977 


i = 11 





0.1246 


i = 20 


0.2225 







c/c = 5 


< = 33 


= 0.2871, Threshold gap =0.2369 dB 


R+ = 0.8932, Threshold gap =0.1357 dB 



TABLE III 

Bilayer-ExpurgatedLDPC Code for SNR+ = -L4237 dB and SNR_ = -3.2972 dB 



Bilayer-Expurgated LDPC code (C) 








i = 2 


0.2417 


0.0496 


I = 3 


0.1702 


0.0501 


i = 6 


0.1182 





i = 7 


0.0056 


0.1348 


i = 18 





0.1573 


i = 19 


0.0267 


0.0458 


7] = 0.9435, 4 = 6, 4 = 2 


= 0.2753, Threshold gap=0.4612 dB 


R+ = 0.3856, Threshold gap=0.2162 dB 



TABLE IV 

Bilayer-LengthenedLDPC Code for SNR_ = -3.0728 dB and SNR+ = -1.4438 dB 



Bilayer-lengthened LDPC code (D) 


Degree 


A! 




i = 2 


0.3227 


0.1580 


i = 3 


0.2107 


0.2045 


i = 5 





0.0461 


i = 6 


0.1247 


0.2171 


I = 7 


0.1194 





i = 12 





0.0058 


i = 13 





0.3685 


i = 20 


0.2225 







dc = 5 


< = 1 


i?_ = 0.2871, Threshold gap =0.2369 dB 


R_^ = 0.3843, Threshold gap =0.2364 dB 
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TABLE V 

Bilayer-ExpurgatedLDPC Code for SNR+ = 2.7330 dB and SNR_ = -0.3273 dB 



Bilayer-Expurgated LDPC code (E) 














I = 2 


0.0998 


0.0805 











I = 3 


0.0827 


0.1331 











i = 6 





0.0086 


0.0920 








i = 7 








0.1725 








I = 20 











0.0845 


0.2463 




r] = 0.8253, d 


c= 15, d 


; = 4 




R 


_ = 0.4618, Threshold gap: 


=0.5143 dB 


R^ 


^ = 0.7000, Threshold gap=0.08474 dB 



TABLE VI 

Bilayer-Lengthened LDPC Code for SNR_ = 0.0229 dB and SNR+ = 2.6122 dB 



Bilayer-lengthened LDPC code (F) 


Degree 


A! 




i = 2 


0.2421 


0.1468 


i = 3 


0.2039 


0.2331 


1 = 6 


0.1677 





% = 7 


0.0829 


0.3039 


i = 8 





0.0298 


i = 19 





0.2864 


I = 20 


0.3034 







dc = 8 


d'=6 


R_ = 0.4877, Threshold gap=0.1641 dB 


R+ = 0.6906, threshold gap=0.1208 dB 



in transmission of the source message to subsequent relays and to the destination. The capacity of this 
decode-and-forward strategy has been studied in [6] and [4]. However, this is not the only possibility. In 
[13], the authors cast the multiple-relay network within a parity-forwarding framework, and have been 
able to enlarge the decode-and-forward rate of [6] and [4]. This section focuses on two-relay networks and 
illustrate two fundamental ways that multiple relays can help each other and help the ultimate decoding 
of information at the destination. The main purpose of this section is to show that practical bilayer codes 
can be readily applied in these cases. 



A. Cascade Bilayer Codes for Two-Relay Networks 

Consider a two-relay network depicted in Fig. 12 In this case, the first relay decodes the message 
from the source Wi, then sends out a parity s], just as in the single-relay case. However, suppose that 
the channel from the source to the second relay is weak. So, the second relay is not able to decode the 
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Fig. 11. Empirical bit error probability curves for the designed codes. Solid straight lines represent Shannon limits for each code, and 
dashed lines represent the convergence threshold computed by density evolution. 



source's message (even with the help of sj), although it is able to decode sj itself. However, for this 
channel, the second relay may still help the ultimate decoding at the destination by sending out parities 
of parities, denoted here as sf, to help the destination decode sj. This "helping-the-helper" strategy can 
be shown to be capacity-achieving for a doubly degraded network [13], and it enlarges the achievable 
rates in [6] and [4]. 

The code construction for this relay network is shown in Fig. 13. It consists of a cascade of two bilayer 
codes. The source message is coded by a bilayer code Ci. Upon decoding Ci, the first relay computes 
additional parities for Ci and re-encodes them using C2, which is another bilayer code. The second relay 
decodes C2, then computes extra parities for C2 and re-encodes them using C3. Finally, the destination first 
decodes C3 to recover the extra parities needed to decode C2. Then, it decodes C2 to recover the parities 



36 




Fig. 12. A two-relay network in which the second relay facilitates the transmission of parity bits from the first relay to the destination 




Fig. 13. Cascaded bilayer codes for the two-relay network in Fig. 12 

of Ci. Finally, the destination decodes Ci. 

Clearly, the bilayer codes that have been devised for single-relay channels can be directly cascaded to 
design coding systems capable of approaching the best achievable rate in this network. 

B. Doubly Bilayer Codes for Two-Relay Networks 

Consider a different two-layer network depicted in Fig. 14 in which the channel between the first relay 
and the destination is weak. In this case, the optimal strategy is for the first relay to help the second relay, 
so that the second relay can ultimately help the destination. 

The code construction for this relay network is shown in Fig. 15. It is a doubly bilayer code in the 
following sense. The source encodes its message using Ci. The first relay decodes Wi, computes k2 parities 
bits, and re-encode the parities using C2 for the second relay. The second relay decodes Wi with the help 
of k2 parities. Then, it computes separate parity bits to be re-encoded by C3. The destination decodes 
C3, then Ci, the source message. The achievable rates using the above strategy is a special case of the 
achievable rate in [6]. 
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Fig. 14. A two-relay network in which the first relay helps the second relay to decode the source message 




Fig. 15. Doubly bilayer codes for the two-relay network in Fig. 14 

For this relay network, C2 and C3 are conventional LDPC codes. However, Ci must be specially designed 
as two bilayer codes extended from the same base code. The code design methodology described in the 
previous section can again be used for this network. For example, Code A and Code E can be utilized to 
construct codebooks for implementing this protocol with a source rate ofR = 0.7. The first relay decodes 
the source codeword at i? = 0.7; the second relay, with the help of k2 parity bits from the first relay at 
a rate 0.7 — 0.6363, can use Code A to decode the source codeword. The second relay then sends out /cs 
parity bits to the destination at a rate 0.7 — 0.4618, which enables the destination to decode the source 
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codeword using Code E. 

VIII. Concluding Remarks 

Binning is of fundamental importance in multiuser information theory. This paper provides a practical 
implementation of the binning strategy for the relay channel from a linear coding perspective, in which 
extra information is generated at the relay to facilitate the overall communication between the source 
and the destination. A key feature of the code design is the construction of a bilayer LDPC code that is 
capable of approaching the Gaussian channel capacity at two different SNRs and at two different rates. 
We show that conventional code design techniques must be significantly modified for the design of these 
multirate codes in order to achieve capacity-approaching performances. 

The code construction in this paper shows that the binning operation for the relay channel is funda- 
mentally easier to implement in practice than the binning techniques for sources and channels with side 
information. The former is an error-correcting problem; the latter essentially a quantization problem for 
which efficient coding methods are not yet known. 

The concept of bilayer codes can be extended to relay networks in which cascades of bilayer codes and 
multilayers LDPC code may be needed. While in principle these codes can be designed and optimized 
for a given network topology, as the network size grows, the encoding and decoding protocols become 
increasingly complex, and the tuning of the code parameters increasingly involved. The code structure 
illustrated in this paper suggests that practical protocols for the relay network should involve universal 
and rateless codes. The bilayer code design methodology described in this paper is a first step toward this 
goal. 
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